1. Field of the Invention
The present invention generally relates to distributed systems. More particularly, embodiments provide client-server systems for efficient handling of client requests.
2. Description of the Related Art
Generally, a distributed computer system comprises a collection of loosely coupled machines (mainframe, workstations or personal computers) interconnected by a communication network. Through a distributed computer system, a client may access various servers to store information, print documents, access databases, acquire client/server computing or gain access to the Internet. These services often require software applications running on the client's desktop to interact with other applications that might reside on one or more remote server machines. Thus, in a client/server computing environment, one or more clients and one or more servers, along with the operating system and various interprocess communication (IPC) methods or mechanisms, form a composite that permits distributed computation, analysis and presentation.
In client/server applications, a “server” is typically a software application routine or thread that is started on a computer that, in turn, operates continuously, waiting to connect and service the requests from various clients. Thus, servers are broadly defined as computers, and/or application programs executing thereon, that provide various functional operations and data upon request. Clients are broadly defined to include computers and/or processes that issue requests for services from the server. Thus, while clients and servers may be distributed in various computers across a network, they may also reside in a single computer, with individual software applications providing client and/or server functions. Once a client has established a connection with the server, the client and server communicate using commonly known protocols (e.g., TCP/IP) or a proprietary protocol defined and documented by the server.
In some client-server implementations sockets are used to advantage. A socket, as created via the socket application programming interface (API), is at each end of a communications connection. The socket allows a first process to communicate with a second process at the other end of the communications connection, usually on a remote machine. Each process communicates with the other process by interacting directly with the socket at its end of the communication connection. Processes open sockets in a manner analogous to opening files, receiving back a file descriptor (specifically, a socket descriptor) by which they identify a socket.
Typically, a simple server establishes a number of active sockets connectors to clients and uses an input/output (I/O) command (e.g., select ( )) to determine when requests arrive from a client. Accordingly, a typical request to the sockets layer for processing a single client request would first determine which server socket has received a request from a client (e.g., using select( )).
Sockets and other client-server mechanisms are shown in the server environments 100 and 200 of FIG. 1 and FIG. 2, respectively. FIG. 1 illustrates synchronous processing and FIG. 2 illustrates asynchronous processing. In general, FIG. 1 shows server environment 100 comprising a main thread 102 and a plurality of worker threads 104. An initial series of operations 106 includes creating a socket (socket( )), binding to a known address (bind( )) and listening for incoming connections on the socket (listen ( )). An accept operation 108 is then issued to accept a new client connection, which is then given to one of the worker threads 104. The operations for accepting a new client connection and giving the client connection to a worker thread define a loop 110 which is repeated until the server is shut down.
Upon taking the client connection from the main thread 102 the worker thread 104 issues a receive operation 112. This operation is repeated (as indicated by loop 114) until the full request is received. The request is then processed and a response is sent using a send operation 116. A loop 118 causes processing to repeat the receive operations 112, thereby handling additional requests from the current client. The worker thread 104 may then take another client connection from the main thread 104 as represented by loop 120.
In general, two approaches are known for receiving requests. In a first approach a receive operation is needed to determine the length of the incoming client request. The length is determined by a server application by reading a length field of the client request. The length field is typically a 2 or 4 byte length at the beginning of the datastream specifying how long the actual request is. The client request then follows the length field in the datastream. Thus, subsequent receives (at least one) from the server application to the sockets layer are needed to receive the actual request data. In a second approach, a datastream format of client requests has one or more terminating characters specifying the end of the request. The typical flow for processing these client requests, includes issuing an input (receive) operation and examining the returned data for the terminating characters. If the terminating characters haven't been received, another input operation is issued. Input operations continue to be issued until the terminating characters ar received. Thus, receive operations are repeatedly issued for some number of bytes until the terminating characters are received. Accordingly, in most cases, both approaches require at least two receives. This repetition of input operations is represented in FIG. 1 by loop 114.
One problem with the latter approach (terminating characters) is that it is difficult to predict how many input operations will be needed. One solution to this problem is to provide the server with buffers sizes that are large enough to handle the maximum client request size. However, this is often a significant waste of resources and does not scale well to a large number of clients.
Alternatively, some server platforms provide a set of asynchronous I/O functions to allow the server design to scale better to a large number of clients. While these implementations vary across platforms, most support asynchronous read and write operations, and a common wait or post completion mechanism. The server applications provide buffers to be filled or emptied of data asynchronously. The status of these asynchronous I/O operations can be checked at a common wait or can be posted back to the application via some mechanism such as a signal. This I/O model can allow a pool of threads to scale to process a much larger set of clients with a limited number of threads in the server application's thread pool.
As an illustration, consider the server environment 200 which uses asynchronous I/O consisting of one main thread 202 accepting client connections and multiple worker threads 204 processing client requests received by the main thread 202. An initial series of operations 206 are the same as those described above with reference to synchronous processing (FIG. 1). Processing of a client request begins when the main thread 202 requests a connection from a client by issuing an asynchronous accept operation 208 for a new client connection to a pending queue 209. Each asynchronous accept operation 208 results in a separate pending accept data structure being placed on the pending queue 209. Once a client connection is established, the appropriate pending accept data structure is removed from the pending queue and a completed accept data structure is placed on a completion queue 210. The completed accept data structures are dequeued by the main thread 202 which issues an asynchronous wait for which a wakeup operation is returned from the completion queue 210. An asynchronous receive operation 214 is then started on a client connection socket 217 for some number of bytes by configuring the pending queue 209 to queue the pending client requests. The number of bytes may either be determined according to a length field which describes the length of the client request or, in the case of terminating characters, for some arbitrary number. The foregoing processing is then repeated, as represented by the loop 215.
Each asynchronous receive operation 214 results in a separate pending receive data structure being placed on the pending queue 209. When a receive completes (the complete client record has been received), the appropriate pending receive data structure is removed from the pending queue 209 and a completed receive data structure is placed on the completion queue 216. An asynchronous wait 218 is issued by a worker thread 204A for which a wakeup operation 220 is returned from the queue 216 with the data.
In the case where a length field is used, the specified number of bytes from the length field is used by the worker thread 204A to issue another asynchronous receive operation 222 to obtain the rest of the client request which is typically received incrementally in portions, each of which is placed in an application buffer. The second asynchronous receive operation 222 is posted as complete to the queue 216 upon receiving the full request and the same or another thread from the thread pool 204 processes the client request. This process is then repeated for subsequent client requests. Where a terminating character(s) is used, each incoming request is dequeued from the queue 216 and checked for the terminating character(s). If the character(s) is not found, another asynchronous receive operation 222 is issued. Asynchronous receive operations are repeatedly issued until the terminating character(s) is received. This repetition for both length field and terminating character implementations is represented by loop 224 in FIG. 2.
Accordingly, each of the foregoing implementations results in a higher than desirable degree of inefficiency. Specifically, the server application uses at least two input operations to obtain the full client request before processing it. Where record headers are used, a first operation is used to obtain the request length and a second operation is needed to process the request. In the case of terminating characters, an even greater number of receive operations may be needed. Where the number of input operations is reduced by pessimistically allocating a buffer large enough to hold the largest possible request, memory may be inefficiently utilized.
Therefore, a need exists for limiting the number of input operations on client requests while efficiently managing memory resources.